The rapid evolution of information technology in recent years, hand in hand with the commercial availability of powerful and sophisticated microprocessors had resulted in the increasing use of parallel processors.
The parallel processing architecture involves the use of many interconnected processors to access large amounts of data. In a massively parallel processing system (MPP system, hereinafter), a relatively large number of, separate though relatively simple, microprocessor based processing elements are interconnected to simultaneously process a large number of tasks at speeds far exceeding those of conventional computers. Though the MPP system is composed of many nodes, the nodes are viewed and function in many ways as one single resource. The grouping of all nodes into a single resource creates advantages in increased capacity and speed. However, this grouping has several key limitations which are of immediate concern.
A first limitation deals with the issue of workload management. Often the users of most large MPP systems require the ability to customize different sets of nodes with different combinations of software in order to manage workload interference. For example, n nodes may be used for processing a parallel data base system, while the remaining nodes are to be used for processing a serial batch application. A parallel processor can run many jobs at once; but each job competes for node and network resources. This is particularly of interest where a switch, such as a High Performance Switch (hereinafter an HPS) is being used. In such instances it is possible for one job to monopolize the switch and thus impact another job using the same switch. Therefore, a need arises for isolating the switch traffic in one environment from the switch traffic in another environment.
Furthermore, many of the MPP systems at present require that all nodes run on the same version of code, also using the same operating system and support programs, which makes customization and workload management difficult, if not impossible.
Using the same version of the code, the same operating system and same support programs can also create migration concerns. This "sameness" requirement can make upgrading the system to a new level of code, a potentially long and risky task, because the users are often forced to upgrade each node to the new levels at the same time. In addition, the users are often forced to install new levels of related software tools as well as their own applications on each node. When and if an error ensues, the users need to reinstall the old levels of code back on each and every affected node and perhaps the entire system, a process known as backing off the new levels of the upgrade.
Furthermore, in many instances users require the capability of testing new levels of software, (such as AIX and LPPs) user applications, and tools, on a system which is currently running a workload without disrupting the production workload. It is, therefore, desirable for the users to test new levels of code on a small number of nodes, while other nodes are unaffected, to avoid disrupting the entire production system. This is particularly important when the new levels of code or upgrade are unfamiliar to the user in their functionality, and perhaps incompatible to other already installed applications.
Akin to the non-disruptive migration requirements, users may require the capability to create multiple production environments with the same non-interfering characteristics. These environments must be sufficiently isolated so that the workload in one environment is not adversely affected by the workload in the other, especially for services whose usage is not monitored and charged for, but which have critical implications on job performance.
As a consequence, in all of the above mentioned situations and other related ones, it would be desirable to be able to "carve out" parts of the system that can run jobs without interfering with each other. This "carving out" process is known as partitioning, as appreciated by those skilled in the art. Partitioning in general is the ability to divide up system resources into groups or parts in order to facilitate particular management functions. The structure of the MPP system provides the opportunity to partition the system into groups of nodes for various purposes.
While there are capabilities at present to perform and maintain successful partitioning in MPP systems, tools are still needed that can easily perform all aspects easily and simultaneously, namely the ones mentioned earlier--workload management, non-disruptive migrations and management of multiple production environments. For example, LoadLeveler program of International Business Machines (IBM) Corp. currently addresses some requirements for workload partitioning, but several other concerns remain unaddressed. In particular, tools are needed to easily form and manage installation of a group of nodes as a single entity, while allowing each partitioned unit to maintain its unique characteristics.
Furthermore, any attempt to partition an MPP system must be designed such that it particularly protects system software and applications from unnecessary changes, while presenting a consistent system model for new code. In addition, the users' need to create multiple production environments with the same non-interfering characteristics that are sufficiently isolated (so that one environment does not adversely affect the working of other environments), needs to be addressed.
The present application is related to the following applications filed on the same day--U.S. application Ser. Nos. 08/664,900, 08/664,577, and 08/664,689.